Connections among basic statistical models

Hover over the links for details on some connections. Dots indicate ‘special cases of’ the parent node. Undirected connections suggest equivalence (≡ on hover) or a more complex relationship (usually for more general/more complex models). Note this is for a quick reference, not an exhaustive one.

Overview

One can start with simple settings of two variables x and y, assuming y is the dependent variable. To examine the relationship we might look at the correlation if they are numeric. If we run a regression by first standardizing our x and y variables, the resulting coefficient for x is equal to the Pearson correlation coefficient. If x is a binary factor, we might use a t-test to examine the group means. In the regression, the coefficient for x would equal the mean difference, and its t-statistic would be identical. ANOVA extends the t-test to more than two groups and more than one variable, but is equivalent to standard regression with categorical predictors. Repeated measures ANOVA extends standard ANOVA to deal with multiple measures per observation, but is a special case of mixed models. Generalized linear models extend the standard linear model to deal with distributions other than normal, and mixed models extend them to deal with dependent observations (e.g. due to repeated measures, but not restricted to that situation). Also included are some traditional multivariate techniques; these really shouldn’t be used anymore, but may provide additional context in case one has come across them.

Labels

Correlation: simple Pearson correlation

Canonical correlation: Canonical correlation

t-test: standard t-test

ANOVA: analysis of variance

MANOVA: multivariate ANOVA

DFA: discriminant function analysis

Repeated Measures: ANOVA models with repeated measures and/or ‘mixed’ design

Regression: standard linear model

GLM: generalized linear models

Mixed Models: generalized linear mixed models

References